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Abstract 


The  measurement  of  shape  is  a  basic  object  inspection  task.  We  use  a  noncontact  method  to 
determine  shape  called  photometric  stereo.  The  method  uses  three  light  sources  which 
sequentially  illuminate  the  object  under  inspection  and  a  video  camera  for  taking  intensity 
images  of  the  object.  A  significant  problem  with  using  photometric  stereo  is  determining 
where  to  place  the  three  light  sources  and  the  video  camera.  In  order  to  solve  this  problem, 
we  have  developed  an  illumination  planner  that  determines  how  to  position  the  three  light 
sources  and  the  video  camera  around  the  object.  The  planner  determines  how  to  position 
light  sources  around  an  object  so  that  we  illuminate  a  specified  set  of  faces  in  an  efficient 
manner,  and  so  that  we  obtain  an  accurate  measurement.  We  predict  the  uncertainty  in  our 
measurements  due  to  sensor  noise  by  performing  a  statistical  simulation  in  our  planner.  This 
gives  us  the  capability  to  determine  when  a  measured  shape  differs  in  a  statistically  signifi¬ 
cant  way  from  what  we  expect.  From  a  high  level,  our  planner  has  three  major  inputs:  the 
CAD  model  of  the  object  to  be  inspected,  a  noise  model  for  our  sensor,  and  a  reflectance 
model  for  the  object  to  be  inspected.  We  have  experimentally  verified  that  the  plans  gener¬ 
ated  by  the  planner  are  valid  and  accurate. 
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1.  Introduction 


The  measurement  of  shape  is  a  basic  object  inspection  task.  This  measurement  is  performed 
in  many  modern  manufacturing  environments.  However,  the  measurements  are  usually 
made  manually.  Manual  inspection  is  monotonous,  is  very  labor  intensive,  and  is  subject  to 
human  error.  Some  companies  have  turned  to  computer  vision  techniques. 


Computer  vision  research  has  produced  a  number  of  basic  techniques  for  measuring  the 
shape  of  an  object:  stereo  vision,  range  finders,  and  photometric  techniques.  Photometric 
techniques,  unlike  stereo,  determine  shape  without  needing  to  solve  the  correspondence 
problem.  Photometric  systems,  unlike  range  finders,  are  cheap  and  are  able  to  measure 
shape  at  a  wide  range  of  resolutions.  For  these  reasons,  we  use  photometric  methods  to 
determine  object  shape.  Photometric  techniques  use  physically  based  reflectance  models 
[14]  to  transform  image  brightness  into  shape.  Image  brightness  depends  on  lighting  geome¬ 
try,  imaging  geometry,  and  surface  shape.  If  we  can  control  imaging  geometry  and  illumina¬ 
tion  geometry,  we  can  use  a  reflectance  model  in  conjunction  with  measured  image 
brightness  to  determine  surface  shape. 


Imaging  geometry  has  been  explored  by  a  number  of  researchers.  However,  very  few 
researchers  have  investigated  how  to  illuminate  an  object  that  they  are  trying  to  inspect.  In 
photometric  stereo,  the  position  of  the  light  souree  affects  what  parts  of  the  object  are  illumi¬ 
nated  and  the  accuracy  with  which  you  can  recover  the  object’s  shape.  The  problem  is  to 
determine  the  best  place  to  put  light  sources  in  order  to  inspect  a  given  object.  If  we  arrange 
light  sources  in  the  shape  of  a  tessellated  sphere  (say  a  N-frequeney  ieosahedron),  the  opti¬ 
mal  light  source  positions  are  not  obvious,  and  the  number  of  potential  light  souree  combi- 
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Fig.  1.  Experimental  Setup 


We  investigate  the  illumination  problem  from  two  perspectives.  First,  we  determine  how  to 
position  light  sources  around  an  object  so  that  we  illuminate  a  specified  set  of  faces  in  an 
efficient  manner.  In  order  to  solve  this  problem,  we  have  to  determine  light  source  visibility 
and  we  have  to  find  some  method  of  efficiently  “covering”  the  specified  set  of  object  faces. 
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Secondly,  we  determine  how  to  position  the  light  sources  so  that  they  give  us  an  accurate 
measurement  of  shape.  There  are  two  basic  types  of  errors  in  photometric  measurements  of 
lambertian  objects:  random  errors  (noise)  and  fixed  errors.  Random  errors  are  due  to  the 
variance  of  the  camera  and  digitizer.  These  are  the  errors  that  we  try  to  predict  with  our 
planner.  Fixed  errors  include:  errors  in  light  source  direction,  errors  in  light  source  radiance, 
errors  in  the  photometric  function.  Fixed  errors  can  be  accounted  for  by  a  careful  calibration 
procedure. 

Noise  causes  uncertainty  in  our  shape  measurement.  The  amount  of  uncertainty  that  this 
noise  will  create  in  our  measurement  will  depend  on  light  source  positions.  By  using  a  noise 
model  of  the  CCD,  a  CAD  model  of  the  object  we  are  inspecting,  and  the  lambertian  reflec¬ 
tance  model  we  can  determine  how  much  uncertainty  we  can  expect  in  our  shape  measure¬ 
ment  for  a  given  light  source  configuration.  We  can  find  an  optimal  light  source 
configuration,  one  that  produces  a  minimum  amount  of  measurement  uncertainty. 

Our  ability  to  predict  the  uncertainty  in  our  measurements  gives  us  the  capability  to  deter¬ 
mine  when  a  shape  differs  in  a  statistically  significant  amount  from  what  we  expect.  This  is 
an  essential  requirement  for  an  inspection  system  because  it  allows  defects  to  be  reliably 
identified. 

The  environments  we  will  study  are  structured.  We  will  assume  that  we  know  what  we  are 
looking  at  (i.e.:  We  have  a  CAD  model  of  the  object  that  we  want  to  inspect. )  and  that  we 
know  the  pose  of  the  object.  This  gives  us  a  tremendous  amount  of  information.  We  can  plan 
our  light  placement  to  sequentially  illuminate  the  entire  surface  of  the  object  (cover  the 
object)  we  are  trying  to  inspect,  and  we  can  use  our  a  priori  geometric  knowledge  of  the 
object  to  optimize  our  inspection  parameters.  This  is  a  realistic  scenario  for  many  structured 
environments  including  manufacturing  environments,  nuclear  plant  maintenance,  and  space 
station  missions. 

The  objects  we  will  look  at  are  convex,  or  are  simple  concavities.  We  assume  orthographic 
projection  and  parallel  incident  light. 

Our  work  allows  surface  shape  measurements  to  be  made.  The  intent  is  that  this  information 
can  be  used  for  inspection.  However,  we  do  not  become  involved  with  setting  up  specific 
inspection  criteria.  Each  inspection  task  requires  unique  inspection  criteria,  which  must  be 
determined  on  a  case  by  case  basis. 

From  a  high  level,  our  problem  has  three  major  inputs:  the  CAD  model  of  the  object  we  are 
trying  to  inspect,  the  noise  model  of  our  sensor  (the  noise  model  for  our  CCD  camera),  and 
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the  reflectance  model  of  the  object  we  are  trying  to  inspect.  These  models  are  used  to  gener¬ 
ate  an  illumination  plan. 


page  4 


2.  Previous  Work 

We  summarize  related  CAD  based  inspection  work.  We  cover  work  by  Cowan,  Tarabanis 
and  Tsai,  Sakane,  Yi,  the  SPIE,  Murase  and  Nayar. 

2.1.  Cowan 

Cowan  developed  the  synthetic  approach  to  determining  a  camera’s  viewpoint.  In  [1],  given 
a  camera  and  lens,  Cowan  develops  methods  for  determining  3D  camera  locations  that  sat¬ 
isfy  the  following  requirements:  focus,  field  of  view,  visibility,  view  angle,  and  prohibited 
regions.  For  each  requirement  he  builds  a  3D  region  that  satisfies  the  requirement.  Then,  he 
intersects  the  regions  to  find  camera  locations  that  satisfy  all  the  requirements. 

The  visibility  constraint  is  the  most  relevant  constraint  to  our  work.  Cowan  determines  visi¬ 
bility  using  the  concept  of  “separating  support  planes”.  Given  a  convex  polygon  “S”  and  an 
occluding  convex  polygon  “O”,  a  separating  support  plane  divides  space  into  two  halves. 
One  half  space  contains  S  and  not  O.  The  other  half  space  contains  O  and  not  S.  The  half 
planes  are  constructing  by  rotating  a  plane  about  each  edge  of  S.  The  plane  is  oriented  so  it 
is  between  O  and  S,  overlapping  S.  Then,  the  plane  is  then  rotated  about  each  edge  of  S  until 
it  hits  a  vertex  or  an  edge  of  O.  This  is  repeated  for  each  edge  of  S,  and  for  each  edge  of  O. 
The  union  of  all  the  half  spaces  containing  S  forms  the  set  of  viewpoints  where  O  does  not 
occlude  S.  The  procedure  can  be  extended  to  convex  polyhedral  objects  and  obstacles. 


In  [2],  Cowan  discusses  positioning  point  light  sources  for  scene  illumination.  First,  given  a 
viewpoint  region,  Cowan  finds  the  minimum  and  maximum  camera  apertures  that  bound  the 
viewpoint  region.  Then,  Cowan  relates  image  irradiance  to  scene  radiance  and  scene  radi¬ 
ance  to  lambertian  scene  irradiance.  Assuming  a  camera  with  a  certain  dynamic  range,  and  a 
light  source  with  a  certain  flux,  he  determines  the  minimum  and  maximum  distance  from  the 
surface  that  the  light  source  can  be  positioned. 

In  [3]  Cowan  reviews  his  previous  work  and  extends  it  by  considering  the  edge  contrast  of 
convex  lambertian  surfaces.  He  determines  the  possible  source  orientations  and  distances, 
with  respect  to  a  given  convex  corner  (consisting  of  two  lambertian  surfaces),  that  produce  a 
required  intensity  difference  (contrast)  between  the  two  surfaces. 
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2.2.  Tsai  and  Tarabanis 

In  [4],  Tsai  and  Tarabanis  develop  a  method  for  determining  the  visibility  regions  for  gen¬ 
eral  concave/convex  polyhedral  targets  occluded  by  general  concave/convex  polyhedral 
obstacles.  Their  method,  at  its  heart,  is  similar  to  Cowan’s  method.  They  have  added  on  a 
convex  decomposition  algorithm,  and  have  improved  the  computational  efficiency  of  Cow¬ 
an’s  original  algorithm. 

In  [5],  Tarabanis,  Tsai,  and  Allen  use  the  methods  of  [4]  to  search  for  visible  regions  where 
a  feature  can  be  viewed.  Then,  they  determine  the  admissible  camera  locations  where  a  fea¬ 
ture  can  be  resolved  to  a  desired  resolution.  Locations  are  determined  for  orthogonal  and 
general  viewing  directions. 

In  [6],  Tarabanis,  Tsai,  and  Allen  expand  on  their  previous  work.  They  want  to  determine 
camera  pose  and  optical  settings  so  that  a  polyhedral  object  is  visible,  in  the  field  of  view,  in 
focus,  and  resolved  to  sufficient  resolution  (four  constraints).  They  pose  the  problem  as  an 
optimization  problem  in  8  dimensional  space.  The  camera  orientation  and  position  contain 
five  degrees  of  freedom.  The  distance  from  the  back  nodal  point  to  image  plane,  lens  focal 
length,  and  aperture  are  the  three  other  degrees  of  freedom.  Visibility  is  determined  using 
the  method  described  in  [4].  The  other  constraints  are  posed  as  inequalities,  and  are  merged 
into  one  optimization  function  with  arbitrary  weights  assigned  to  each  constraint.  The  func¬ 
tion  is  then  optimized  using  the  visibility  region  boundaries  as  optimization  constraints. 

Tsai  and  Tarabanis  and  Cowan  use  a  synthesis  approach  to  sensor  location.  The  sensor  loca¬ 
tions  that  satisfy  a  task  are  determined  directly  using  optimization  techniques.  Sakane  and 
Yi  use  a  generate  and  test  approach.  Sensor  locations  are  generated.  Then,  they  are  evalu¬ 
ated  against  a  criteria  function. 

2.3.  Sakane 

In  [9],  Sakane  describes  a  system  that  determines  camera  placement  on  a  geodesic  dome, 
that  is  placed  around  a  target’s  point  of  interest.  The  camera  is  placed  so  that  the  point  of 
interest  is  not  occluded  by  other  objects  in  the  environment  or  by  the  manipulator  holding 
the  camera.  Sakane  determines  occlusion  free  regions  by  doing  depth  buffering  from  the 
center  of  the  dome  to  each  facet.  The  minimum  distance  is  stored  at  each  facet.  If  the  mini¬ 
mum  distance  is  less  than  the  radius  of  the  dome,  the  viewpoint  is  occluded.  The  radius  of 
the  geodesic  dome  is  chosen  to  get  sufficient  target  resolution. 

In  [10,  11],  Sakane  describes  positioning  light  sources  for  a  photometric  stereo  system.  He 
tries  to  optimize  the  extracted  surface  normal  and  the  surface  coverage  for  lambertian  sur¬ 
faces.  Sakane  uses  the  method  of  determining  occlusion  free  viewpoints  described  in  [9]  to 
find  candidate  positions  for  the  light  sources.  These  positions  provide  shadow  free  illumina¬ 
tion. 

Sakane  proposes  a  metric  of  surface  orientation  reliability  that  relies  on  the  condition  num¬ 
ber  of  the  source  matrix  to  estimate  the  error  in  the  surface  normal  vector  (By  definition,  iNl 
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The  second  metric  Sakane  uses  is  the  size  of  the  intersection  region  of  the  three  light  sources 
and  the  camera  on  the  gaussian  sphere.  Each  light  source  will  illuminate  a  hemisphere  of  the 
Gaussian  sphere,  and  the  camera  will  be  able  to  observe  a  hemisphere  of  the  gaussian 
sphere.  By  intersecting  the  four  hemispheres,  Sakane  determines  the  amount  of  the  gaussian 
sphere  that  is  detectable. 

Sakane  combines  the  detectability  metric  and  surface  orientation  reliability  metric  into  a  sin¬ 
gle  criterion  function.  The  weights  of  the  two  metrics  are  arbitrary.  In  [11],  Sakane  uses  a 
movable  camera.  This  allows  a  further  degree  of  freedom  in  optimizing  the  detectability 
metric.  In  [12],  Sakane  considers  lambertian  edge  contrast  as  a  metric.  Visibility  seems  to  be 
determined  using  the  methods  described  in  [9]. 

2.4.  Yi 

In  [13],  Yi  considers  edge  contrast  for  specular  lobe  objects,  using  a  polarized  light  version 
of  the  Torrance  Sparrow  model.  Yi  forms  a  discrete  spherical  viewing  space,  with  points 
positioned  so  that  the  arc  length  between  viewing  points  is  approximately  equal.  At  discrete 
points  along  an  edge  Yi  calculates  the  intensity  difference.  Then,  he  finds  the  “contrast  dis¬ 
tribution  function”  metric,  which  tells  how  much  of  the  edge  has  a  contrast  above  a  certain 
threshold.  The  second  metric  is  sensor  visibility  of  the  edge,  which  he  defines  as  the  ratio  of 
the  unoccluded  portion  of  the  edge  to  the  entire  edge  length.  Given  a  set  of  required  edges, 
Yi  searches  for  light  source  and  sensor  position  which  maximizes  the  two  criteria  functions. 
Yi  does  not  describe  how  he  combines  the  two  criteria  functions.  The  results  presented  in 
the  paper  are  for  a  synthetic  cylinder  and  cube.  No  real  results  are  presented. 

2.5.  SPIE  -  Society  of  Photo-Optical  Instrumentation  Engineers 

The  SPIE  (Society  of  Photo-Optical  Instrumentation  Engineers)  has  a  large  literature  [23  - 
25]  on  machine  vision.  Their  approach  is  to  solve  real  industrial  problems.  They  have  real¬ 
ized  the  importance  of  illumination  for  detecting  defects  in  surfaces.  However,  their 
approach  is  best  described  as  expert  experience.  For  each  inspection  task,  they  formulate  an 
illumination  strategy,  based  on  their  own  trial  and  error  experience  [23].  They  have  even 
developed  expert  systems,  which  give  illumination  strategies  based  on  task  specifications 
[24].  For  example,  one  application  is  to  view  bright  metal  surfaces  without  causing  glinting. 
Their  solution  is  to  use  diffuse  illumination.  While  their  techniques  are  useful,  they  tend  to 
be  adhoc. 

2.6.  Murase  and  Nayar 

Murase  and  Nayar  [37]  have  developed  an  illunfination  planning  system  for  object  recogni¬ 
tion.  They  take  a  sequence  of  images  of  a  set  of  objects,  using  different  light  source  direc¬ 
tions  and  different  object  poses.  Then,  they  project  the  image  set  into  a  universal  eigenspace, 
which  is  a  low  dimensional  representation  of  the  image  set.  Each  object  will  trace  out  a 
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hypercurve  in  the  universal  eigenspace.  The  distance  between  curves  can  be  used  as  a  metric 
to  distinguish  objects.  An  illumination  configuration  that  maximizes  the  distance  between 
hypercurves  can  be  used  to  discriminate  between  objects  in  an  optimum  way. 

2.7.  Summary/Conclusions 

Cowan  addressed  the  issues  of  visibility,  imaging  parameters,  irradiance,  and  lambertian 
edge  contrast.  Tsai  and  Tarabanis  addressed  the  issues  of  visibility  and  imaging  parameters. 
Sakane  addressed  the  issue  of  illumination  in  order  to  determine  the  shape  of  a  lambertian 
object.  Yi  addressed  the  issue  of  finding  specular  lobe  edge  contrast. 

The  problem  of  finding  edge  contrast  is  very  different  than  recovering  surface  shape  using 
photometric  methods.  In  order  to  generate  edge  contrast,  one  only  needs  to  achieve  a  high 
intensity  difference  between  the  two  surfaces  that  form  the  edge.  This  is  very  different  from 
using  intensity  to  find  shape.  Sakane’s  work  is  the  closest  to  our  research.  There  are  at  least 
three  major  differences  between  Sakane’s  approach  and  ours.  First,  we  propose  a  new  metric 
for  finding  orientation  error.  This  is  developed  in  section  3.  Secondly,  Sakane  solves  the 
problem  of  positioning  lights  to  avoid  objects  in  the  environment  or  the  manipulator  holding 
the  camera.  We  are  positioning  lights  so  that  we  illuminate  a  specified  set  of  object  faces  in 
an  efficient  manner.  Thirdly,  we  incorporate  an  accurate  sensor  model  in  our  solution. 
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3.  2D  Convex  Lambertian  Illumination 

Lambertian  illumination  is  important  for  two  reasons.  First,  the  illumination  of  lambertian 
surfaces  is  a  basic  problem.  Lambertian  surfaces  are  one  of  the  three  basic  surface  types. 
(Lambertian,  specular  lobe,  and  specular  spike  describe  the  basic  surface  types.)  Solving  the 
lambertian  illumination  problem  will  provide  a  foundation  for  approaching  other,  more  dif¬ 
ficult  problems. 

The  2D  problem  presented  here  is  simplified.  However,  it  will  provide  insight  into  the  solu¬ 
tion  space  of  the  lambertian  illumination  problem.  The  solution  space  of  the  3D  lambertian 
illumination  problem,  which  we  present  later  in  this  paper,  is  much  more  complex.  It  is  very 
hard  to  visualize  because  of  its  high  dimensionality. 

3.1.  Introduction 

For  our  initial  investigations,  we  have  chosen  a  simplified  version  of  the  2D  lambertian  illu¬ 
mination  problem.  The  problem  does  not  consider  surfaces,  but  considers  discrete  normals 
in  a  2D  space.  We  concentrate  on  visibility,  minimum  coverage,  and  the  accuracy  of  surface 
normal  recovery.  Our  discrete  normals  exhibit  lambertian  reflectance  characteristics.  Our 
sensor  noise  model  is  a  fixed  amount  of  sensor  noise.  (In  3D  we  will  use  a  geometric  mod¬ 
eler  and  a  planning  system  to  reason  about  surfaces.  We  will  also  use  an  accurate  sensor 
noise  model.) 

Our  approach  is  to  identify  the  visibility  range  for  each  normal.  After  we  have  the  visibility 
range  for  each  normal,  we  break  the  viewing  circle  into  visibility  regions.  Within  each  visi¬ 
bility  region,  certain  normals  are  viewable  (This  idea  is  similar  to  an  aspect.)  We  then  try  to 
find  combinations  of  visibility  regions  that  provide  an  exact  cover  of  the  normals  we  are  try¬ 
ing  to  view.  Since  we  are  working  in  2D  lambertian  space,  we  need  two  light  sources  to 
recover  the  normals  within  each  visibility  region.  After  we  have  our  minimum  covers,  we 
determine  the  most  reliable  lighting  positions  for  recovering  the  orientation  of  the  surface 
normals  within  each  visibility  region. 

3.2.  2D  Visibility  Regions 

Given  a  set  of  normals  (nl,  n2,  n3,....),  we  first  find  the  visible  range  for  each  normal.  Since 
these  are  discrete  normals,  we  do  not  need  to  consider  occlusion.  So,  the  visible  range  of 
each  normal  is  simply  +/-  90°  with  respect  to  the  normal’s  orientation.  Once  we  have  calcu¬ 
lated  the  visibility  range  for  each  normal,  we  divide  the  2D  viewing  circle  into  visibility 
regions.  The  regions  are  formed  by  sorting  the  visibility  ranges  of  all  the  normals  into  a  con¬ 
tinuous  list.  The  list  is  then  converted  into  intervals.  Within  each  interval,  certain  normals 
will  be  visible.  For  example: 

If  we  have  normals  at  45°,  90°,  135°,  and  180°  we  have  the  following  visibility 
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2D  Visibility  range/region  diagram: 
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Fig.  4.  Visibility  Range/Region  Diagram 
The  associated  visibility  regions  are: 
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3.3. 2D  Exact  Covers 

After  we  have  the  visibility  regions,  we  find  combinations  of  visibility  regions  that  form 
exact  covers  of  our  set  of  normals  (An  exact  cover  contains  all  of  the  normals,  non-redun- 
dantly.)  Since  the  number  of  regions  is  small,  we  use  exhaustive  search  to  find  the  exact  cov¬ 
ers.  If  “n”  is  the  number  of  visibility  regions,  the  complexity  of  this  search  is  0(2").  (In  the 


next  section  of  the  paper,  we  develop  a  heuristic  for  finding  exact  covers.)  For  the  previous 
example,  the  exact  covers  are: 


Cover  number 

visibility  regions  included 
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3.4. 2D  Orientation  Error 

An  error  in  light  source  illumination  will  cause  an  error  in  surface  normal  recovery.  This  can 
be  seen  from  the  lambertian  equation: 


Six  Sly 

-1 

II 

Nx 

Six  Sly_ 

12 

(Six  and  Sly  are  the  x  and  y  components  of  the  unit  vector  to  light  source  number  one.  Nx 
and  Ny  are  the  x  and  y  components  of  the  unit  surface  normal.  II  is  the  measured  intensity  at 
N  due  to  light  source  number  one.) 


An  error  in  either  II  or  12  will  cause  an  error  in  Nx  and  Ny: 


51a:  51>- 

-1 

/H-8/1 

Nx  +  8Nx 

Six  Sly 

11  +  811 

Ny  +  8Ny 

In  matrix  notation,  we  can  write: 

S"*(/  +  5/)  =  iV+5iV 
We  call  “N  +  5N”  a  “noisy  normal”. 

The  amount  of  disturbance  caused  by  51  depends  on  the  condition  number  of  S  [38].  If  c  is 
the  condition  number  of  S  then: 

|5N|  |6I| 

INI  III 


For  example,  if  we  have  a  normal  at  90°  (The  nominal  value  of  Nx  =  0.0,  and  the  nominal 
value  of  Ny  =  1.0.),  SI  at  100°,  S2  at  80°,  and  5ll  and  612  independently  range  from  -0.1  to 
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0.1,  we  will  get  the  following  noisy  normal  distribution  in  Nx-Ny  vector  space: 


Fig.  5.  Noisy  Normal  Distribution:  N  at  90°,  SI  at  100°,  S2  at  80°. 

The  condition  number  of  S  is  5.7.  The  magnitude  of  I  and  N  are  both  1.0,  and  51  =  0.1. 
Therefore 


|6N| 


111 


<5.7 


10.11 

111 


<0.57 


which  is  the  magnitude  of  the  disturbance  seen  along  Nx  in  the  noisy  normal  distribution. 
So,  the  condition  number  gives  us  a  way  of  predicting  the  magnitude  of  an  expected  error  in 
Nx,  Ny  vector  space. 

The  noisy  normal  distributions  provide  insight  into  the  effect  of  different  source  matrices. 
However  the  noisy  normal  distributions  are  only  one  component  of  the  problem.  Surface 
orientation  is  represented  by  a  unit  normal.  The  normals  in  the  noisy  normal  distribution  are 
not  guaranteed  to  be  unit  normals.  Therefore,  we  normalize  the  resulting  values  of  Nx  and 
Ny.  The  noisy,  normalized,  normal  is: 

Nx  +  ^Nx  Ny  +  bNy 
^ noise  =  +  ’  \n+5N\ 

TV  =  Mx  ■  Nv- 
noise  niose^  -^noise 


When  we  do  this,  we  perform  a  non-linear  transformation  from  Nx-Ny  vector  space  to  the 
unit  normal  circle.  The  angle  between  a  noisy  normal  and  the  nominal  normal  is  the  angular 
orientation  error.  For  our  2D  case,  the  angular  orientation  error  is  worst  at  the  two  extremes 
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of  the  noisy  normal  distribution.  This  is  shown  below: 

Ny 
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-1.0  -0.5  0.0  0.5  1.0  Nx 

Fig.  6.  Normalized  Noisy  Normal  Distribution:  N  at  90°,  SI  at  100°,  S2  at  80° 

There  may  be  little  correlation  between  the  magnitude  of  the  normal  vector’s  error  in  Nx-Ny 
vector  space  and  the  resulting  angular  orientation  error.  This  point  can  be  emphasized  by 
looking  at  a  second  example.  If  we  have  a  normal  at  90°,  SI  at  10°,  S2  at  170°,  and  6ll  and 
512  independently  range  from  -0.1  to  0.1,  we  will  get  the  following  normal  distribution  in 
Nx-Ny  vector  space  (The  nominal  normal  is  the  point  Nx  =  0.0,  Ny  =  1.0.): 

Ny 

1.5 


1.0 


0.5 

0.0 

-1.0  -0.5  0.0  0.5  1.0  Nx 

Fig.  7.  Normalized  Noisy  Normal  Distribution:  N  at  90°,  SI  at  10°,  S2  at  170° 

In  the  two  figures,  the  worst  case  error  in  Nx-Ny  vector  space  is  the  same.  The  noisy  normal 
distribution  in  the  two  figures  is  the  same  except  for  a  90°  rotation.  The  condition  numbers 
of  the  two  source  matrices  is  also  the  same.  However,  normalization  causes  the  angular  ori¬ 
entation  errors  to  be  very  different. 

The  condition  number  combined  with  6l  determine  the  size  of  the  noisy  normal  distribu¬ 
tion’s  major  axis.  The  orientation  of  the  surface  normal  determines  how  the  noisy  normal 
distribution  will  be  projected  onto  the  unit  sphere,  producing  the  angular  orientation  error. 

Sakane  [10,  11]  proposes  a  metric  of  surface  orientation  reliability  that  relies  on  estimating 
the  unnormalized  vector  error.  His  method  uses  the  condition  number  of  the  source  matrix  to 
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estimate  the  error  in  the  surface  normal  vector  (By  definition,  iNl  =  1.): 


16N| 

INI 


<cond(S)  ■ 


1 511 

HI 


As  we  have  just  seen,  there  is  not  necessarily  any  correlation  between  the  magnitude  of  5N 
and  the  resulting  orientation  error. 

Due  to  this  problem  with  Sakane’s  method,  we  developed  another  method  for  optimizing 
light  source  placement  on  lambertian  surfaces.  Our  method  calculates  N  +  5N  for  a  given  S, 
N,  and  61.  We  normalize  the  resulting  N  +  8N  vector  (We  call  the  normalized  N  +  6N  vector 
Nnoise-)-  Then,  we  find  the  angular  orientation  error  between  N  and  Nnoigg. 

^err  =  ^ noise') 


For  the  2D  case,  we  use  the  average  of  the  two  maximal  angular  orientation  errors. 


3.4.1.  Angular  Orientation  Error  Surface  for  One  Normal 

In  order  to  further  develop  our  intuition,  we  next  look  at  the  error  surface  for  a  single  normal 
at  90°. 


Fig.  8.  Error  Surface:  N=90°,  51=0.01 


The  error  is  greatest  when  the  light  sources  are  close  together  and  the  incident  angle 
between  the  light  sources  and  surface  normal  is  small.  The  condition  number  of  the  source 
matrix  is  large  when  the  light  sources  are  close  together.  However,  as  Fig.  9  shows,  the 
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source  matrix  condition  number  also  is  large  when  the  sources  are  very  far  apart. 


Fig.  9.  Source  Matrix  Condition  Number. 


As  in  the  case  of  the  noisy  normal  distributions,  the  condition  number  does  not  explain  the 
secondary  shape  characteristics  of  the  error  surface.  In  order  to  understand  the  error  surface, 
we  need  to  look  at  another  factor.  This  is  the  incident  angle  between  the  light  source  and  the 
normal.  When  the  incident  angle  between  the  light  source  and  normal  is  large,  the  sensitivity 
of  the  normal  to  disturbances  is  small.  When  the  incident  angle  between  the  source  and  nor¬ 
mal  is  small,  the  sensitivity  of  the  normal  to  disturbances  is  large.  This  is  because  of  the 
shape  of  the  cosine  function.  We  are  using  intensity  to  determine  surface  orientation.  A 
change  in  intensity  due  to  noise  will  cause  a  change  in  surface  orientation.  If  we  look  at  the 
cosine  function,  we  can  see  that  at  small  incident  angles,  the  sensitivity  of  the  normal  will  be 
high,  and  at  large  incident  angles,  the  sensitivity  of  the  normal  will  be  much  less. 


3.4.2.  Angular  Orientation  Error  Surface  for  Multiple  Normal 

We  go  back  to  our  exact  cover  example,  where  we  had  normals  at  45°,  90°,  135°,  and  180°. 
We  let  511  and  512  independently  range  from  -0.01  to  0.01,  and  assess  our  ability  to  accu¬ 
rately  determine  surface  orientation  within  each  visibility  range,  of  each  exact  cover.  Each 
visibility  range  covers  certain  normals.  We  move  two  light  sources,  in  one  degree  incre¬ 
ments,  between  the  maximum  and  minimum  visibility  values  of  each  visibility  range.  At 
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each  position,  we  find  the  total  angular  error,  in  degrees,  for  the  normals  within  the  visibility 
range.  After  searching  through  the  whole  visibility  range,  we  find  the  light  source  placement 
with  the  minimum  error. 

We  get  these  results  (Tot_error  is  the  sum  of  the  angular  errors,  in  degrees,  for  the  normals 
within  the  visibility  range.  The  columns  45°,  90°,  135°,  and  180°  list  the  angular  error,  in 
degrees,  for  each  of  these  normals.  Min_sl  and  Min_s2  are  the  light  source  positions  that 
produced  the  minimum  error.): 


cover 

Visibility 

Range 

Tot_error 

45° 

o 

O 

o^ 

135° 

o 

O 

00 

min_sl 

min_s2 

1 

90.0 

135.0 

4.10 

0.60 

1.45 

1.45 

0.60 

91.0 

134.0 

2 

0.0 

45.0 

2.05 

1.45 

0.60 

1.0 

44.0 

2 

180.0 

225.0 

2.05 

0.60 

1.45 

181.0 

224.0 

3 

45.0 

90.0 

3.50 

1.45 

1.45 

0.60 

46.0 

89.0 

3 

225.0 

270.0 

0.6 

0.6 

226.0 

269.0 

4 

135.0 

180.0 

3.50 

0.60 

1.45 

1.45 

136.0 

179.0 

4 

315.0 

360.0 

0.60 

0.60 

316.0 

359.0 

A  tessellated  error  surface  for  cover  1  is  show  below.  The  error  is  greatest  when  the  light 
sources  are  close  together  and  is  least  when  the  light  sources  are  far  apart. 


Fig.  1 1 .  2D  Angular  Orientation  Error  Surface 


3.4.3.  Source  Intensity  Versus  Angular  Orientation  Error 

Another  factor  to  consider  is  the  relationship  between  source  radiance  and  angular  error. 
Below  we  plot  angular  error  for  different  source  matrices  versus  source  intensity.  As  source 
intensity  increases,  the  angular  error  decreases.  This  is  because  the  signal  to  noise  ratio  is 
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larger.  The  noise  produces  less  relative  disturbance  to  surface  orientation. 


0.0  50.0  100.0  150.0  200.0 

Source  Intensity 


Fig.  12.  Angular  Error  Versus  Source  Intensity.  N  =  90°. 
(±3a  noise  model  from  section  6.1) 
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4.  3D  Convex  Illumination  Covers 

Finding  illumination  covers  in  3D  is  similar  to  finding  illumination  covers  in  2D.  However 
the  3D  problem  is  more  complex.  Given  a  3D  convex  polyhedral  object,  and  a  set  of  faces 
on  the  object  that  we  want  to  inspect,  we  want  to  place  light  sources  around  the  object  in  a 
way  that  illuminates  the  set  of  object  faces  that  we  want  to  inspect,  while  minimizing  the 
total  number  of  measurements  we  need  to  make. 

In  order  to  illuminate  an  object  under  inspection,  we  need  to  be  able  to  divide  the  viewing 
sphere  into  aspects.  Each  aspect  describes  topologically  connected  viewing  directions  that 
see  the  same  set  of  object  faces.  Eventually,  aspects  need  to  be  combined  in  some  manner 
that  provides  coverage  of  all  the  object  faces  that  need  to  be  examined.  (We  define  our 
inspection  task  to  be  inspection  of  a  specified  set  of  object  faces.  This  set  may  include  all 
object  faces,  or  it  may  be  a  proper  subset  of  all  object  faces.)  A  combination  of  aspects  that 
views  all  specified  faces  of  an  object  in  a  non-redundant  manner  is  called  an  exact  cover. 
There  are  two  types  of  aspects:  aspects  that  are  formed  from  viewpoints  near  the  object  and 
aspects  that  are  formed  by  viewpoints  far  from  the  object.  For  now,  we  are  concentrating  on 
aspects  that  are  formed  from  viewpoints  far  from  the  object. 

We  generate  CAD  models  of  objects  that  we  want  to  inspect  using  the  “Vantage”  geometric 
modeling  system  [30].  Using  these  models  and  the  3D-to-2D  scene  generation  capabilities 
of  Vantage,  we  can  generate  orthographic  projections  from  various  viewpoints  of  a  tessel¬ 
lated  icosahedron.  By  comparing  the  area  of  each  visible  object  face  with  its  foreshortened 
projected  area,  we  can  determine  which  object  faces  are  completely  visible  from  each  view¬ 
point.  If  the  face  is  more  than  99%  visible,  we  consider  the  face  to  be  completely  visible. 

Once  we  know  which  object  faces  are  visible  from  each  icosahedron  viewpoint,  we  merge 
adjacent  icosahedron  viewpoints  that  view  the  same  set  of  object  faces.  The  resulting 
merged  set  of  icosahedron  viewpoints  forms  an  aspect.  Each  aspect  is  a  continuous  viewing 
region.  A  camera  or  light  source  placed  anywhere  within  the  region  will  view  the  same  set 
of  object  faces. 

Given  the  list  of  faces  visible  from  each  aspect  and  a  list  of  all  the  faces  on  the  object  that  we 
are  inspecting,  we  can  determine  combinations  of  aspects  that  form  exact  covers. 

4.1. 3D  Aspect  Generation 

Koenderink  and  van  Dorn  [15]  described  aspects  and  their  associated  data  structure  the 
visual  potential  (aspect  graph).  Since  then,  many  researchers  [22]  have  explored  aspect 
graphs,  and  their  application  to  computer  vision  tasks.  The  primary  application  of  aspect 
graphs  to  computer  vision  has  been  object  recognition  [16,  17].  Aspect  graphs  are  used  to 
predict  potential  object  appearances.  Then,  sensor  measurements  which  can  recognize  an 
object  based  upon  the  set  of  possible  object  appearances  and  sensor  measurements  can  be 
planned.  There  are  two  types  of  aspect  prediction  possible  exact  aspect  graphs  and  approxi¬ 
mate  aspect  graphs.  Exact  aspect  graphs  [18-21]  use  a  3D  model  of  an  object  to  predict 
where  in  viewpoint  space  visual  events  occur.  Once  the  loeation  of  a  visual  event  is  known, 
viewpoint  space  can  be  partitioned  into  regions.  Each  region  will  correspond  to  an  aspect. 


Exact  aspect  graphs  are  mathematically  complete.  They  describe  every  aspect  that  an  object 
produces,  no  matter  how  small.  However,  producing  an  exact  aspect  graph  for  general 
objects  is  a  non-trivial  problem  which  is  still  an  area  of  active  research. 

The  approximate  aspect  graph  approach  [16,  17]  uses  a  tessellated  viewing  sphere  that  sur¬ 
rounds  the  object.  Views  from  each  viewpoint  on  the  viewing  sphere  are  generated.  Views 
that  are  topologically  equivalent  are  merged  into  aspects.  The  approximate  aspect  graph 
method,  unlike  the  exact  aspect  graph  method,  will  work  on  all  classes  of  objects.  By  using 
this  approach,  we  will  be  able  to  inspect  a  large  class  of  objects. 

The  approximate  aspect  graph  approach  misses  aspects  that  are  close  to  the  object  being 
viewed  (usually  the  viewing  sphere  is  much  larger  than  the  object).  The  method  may  miss 
small  aspects  if  the  tessellation  is  not  fine  enough,  and  the  approximate  aspect  graph 
approach  is  computationally  intensive. 

For  our  application,  aspects  that  are  close  to  the  object  are  not  important.  We  are  trying  to 
perform  a  macroscopic  level  inspection  of  an  entire  object,  not  a  microscopic  inspection  of  a 
partial  face.  Small  aspects  are  undesirable  because  they  are  unstable.  Computational  load  is 
not  issue  because  the  inspection  plan  can  be  generated  off-line. 

4.2.  3D  Exact  Coverage 

The  problem  of  finding  all  exact  covers  of  a  set  of  faces  is  equivalent  to  the  set/subset  exact 
cover  problem.  This  problem  is  known  to  be  NP-complete.  In  order  to  decrease  the  com¬ 
plexity  of  finding  exact  covers,  we  developed  a  heuristic  search  approach  to  the  problem 
Our  heuristic  measure  is;  the  number  of  object  faces  covered  by  the  aspect.  Our  algorithm  is 
as  follows; 

1.  Select  an  object  face. 

2.  Find  the  largest  aspect,  Amax,,  (aspect  that  sees  the  most  object  faces) 
that  contains  this  object  face. 

3.  Delete  any  aspects  that  overlap  Amaxj,  including  AmaX],  from  the 
aspect-list. 

4.  n=l. 

5.  Find  the  largest  remaining  aspect  in  the  aspect-list,  AmaXn. 

6.  See  if  the  combination  of  Amaxj,  Amax2,  ...  ,  Amax^  covers  all  object 
faces.  If  so,  return  the  exact  cover;  Amaxj,  Amax2,  . . ,  AmaXn. 

7.  If  not,  delete  any  aspects  that  overlap  Amax^,  including  Amax^,  from  the 
aspect-list. 

8.  If  the  aspect-list  is  empty,  signal  failure  and  return. 

9.  n  =  n+l. 

10.  Go  to  step  5. 

In  steps  2  and  5,  if  there  are  two  or  more  aspects  that  are  the  largest  aspects,  we  trace  out  all 
of  the  largest  aspects  in  parallel.  We  can  expand  our  criteria  for  finding  the  next  aspect  to 
expand  in  steps  2  and  5  to  include  the  largest  aspects,  the  2nd  largest  aspects,  ...,  the  qth 
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largest  aspects.  As  q  increases,  the  algorithm  will  approach  exponential  complexity. 

The  maximal  aspect  heuristic  was  chosen  because  we  are  trying  to  minimize  the  number  of 
sensor  measurements  needed  to  cover  an  object.  By  choosing  aspects  that  cover  as  many 
object  faces  as  possible,  we  maximize  the  coverage  of  each  sensor  measurement.  This 
should  tend  to  minimize  the  number  of  sensor  measurements  needed. 

We  do  not  offer  a  formal  proof  that  our  algorithm  will  succeed  for  all  objects.  (The  success 
of  the  algorithm  is  dependent  on  the  tessellation  of  the  viewing  sphere  since  some  aspects 
might  be  missed  if  the  tessellation  is  not  fine  enough.)  However,  we  can  say  that  as  our 
search  expands  to  include  the  qth  largest  aspect,  the  aspect  size  will  in  the  worst  case 
approach  the  size  of  a  single  face.  This  is  possible  in  the  case  of  cube,  where  it  is  possible  to 
have  singular  aspects  that  contain  each  cube  face.  Then,  an  exact  cover  can  be  formed  by 
combining  these  single  face  aspects.  As  object  complexity  increases,  the  object  will  eventu¬ 
ally  approach  a  tessellated  sphere.  In  this  case,  two  views,  180°  apart,  provide  an  exact 
cover. 

In  order  to  test  our  algorithm  against  the  set  of  exact  covers,  we  compared  our  level  one 
search  (only  the  largest  aspects),  with  deeper  level  searches  (largest  aspects,  the  2nd  largest 
aspects,  ...,  the  qth  largest  aspects)  for  the  following  objects:. 


Fig.  13.  Convex  Polyhedra  Used  for  Exact  Cover  Tests 


Aspects  are  formed  by  using  an  3-frequency  (320  face)  icosahedron.  The  icosahedron  has  a 
radius  equal  to  1000  times  the  largest  diameter  of  the  object  being  inspected.  Views  are  gen¬ 
erated  using  orthographic  projection. 


page  20 


The  covers  in  Table  1  include  aspects  made  up  of  any  number  of  icosahedron  viewpoints. 
For  photometric  stereo,  we  would  need  to  place  three  light  sources  in  an  aspect.  Therefore, 
any  aspect  would  need  to  be  made  up  of  at  least  three  icosahedron  viewpoints.  The  results  in 
Table  2  are  for  photometric  stereo.  Only  aspects  made  up  of  at  least  3  icosahedron  view¬ 
points  are  allowed  to  be  part  of  an  exact  cover. 


Table  1:  Exact  Covers  -  All  Aspects 


Search  Depth 

cube 

tetrahedron 

truncated 

cone 

double 

truncated  cone 

1 

4 

12 

41 

79 

2 

68 

21 

45 

79 

3 

68 

23 

45 

79 

4 

23 

Table  2:  Exact  Covers  for  photometric  stereo-  Aspects  of  size  3  and  greater. 


Search  Depth 

cube 

tetrahedron 

truncated 

cone 

double 

truncated  cone 

1 

4 

7 

13 

21 

2 

4 

13 

13 

21 

3 

4 

14 

13 

21 

The  reason  that  we  do  not  see  a  jump  in  the  number  of  aspects  for  the  cube  when  we 
increase  the  search  depth  from  2  to  3  is  that  the  icosahedron’s  viewpoints  do  not  line  up  with 
the  cube’s  faces.  So,  we  do  not  have  any  aspects  that  see  only  one  object  face.  We  only  have 
aspects  that  see  two  or  three  object  faces.  The  single  object  face  cube  aspects  are  actually 
accidental  aspects.  They  are  not  really  useful  because  they  occur  at  one  exact  viewing  direc¬ 
tion.  They  are  very  unstable.  In  the  case  of  the  tetrahedron,  we  have  a  couple  of  aspects  that 
see  only  one  object  face.  These  aspects  are  large,  containing  many  icosahedron  viewpoints. 
We  also  have  aspects  that  see  one  object  face,  two  object  faces,  and  three  object  faces. 
Therefore,  we  have  more  jumps  in  the  number  of  aspects  for  the  tetrahedron  than  for  the 
cube. 


I 

I 

! 
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5.  3D  Convex  Lambertian  Illumination 

Light  source  positioning  in  3D  is  similar  to  light  source  positioning  in  2D.  The  goal  is  to 
find  the  positions  for  three  light  sources  that  illuminate  a  given  set  of  object  faces  while  min¬ 
imizing  the  “total  orientation  error”  for  the  object  faces.  An  exact  illumination  cover  for  a 
given  set  of  object  faces  and  a  candidate  set  of  light  source  positions  is  found  using  the  heu¬ 
ristic  developed  in  the  previous  section.  Then,  for  each  illumination  aspect,  three  light 
sources  are  placed  in  all  of  the  combinations  of  light  source  positions  that  comprise  the  illu¬ 
mination  aspect.  The  number  of  combinations  is  “N  choose  3”,  since  we  are  placing  three 
light  sources  in  the  N  light  source  positions  that  comprise  the  aspect.  The  “total  orientation 
error”  is  evaluated  for  each  combination.  The  light  source  positions  that  produce  the  mini¬ 
mum  total  orientation  error  is  returned  by  the  planner. 

5.1.  3D  Orientation  Error 

As  we  developed  in  the  two  dimensional  case,  an  error  in  light  source  illumination  will 
cause  an  error  in  surface  normal  recovery.  This  can  be  seen  from  the  lambertian  equation: 

Sly  sQ  f/l  Nx 
S2xS2yS2z  12  =  Ny 
S3x  S3y  S3zj  L^sJ 

II,  12,  and  13  are  the  mean,  measured,  light  source  intensities.  Noise  in  either  II,  12,  or  13 
will  cause  an  error  in  Nx,  Ny,  and  Nz,  producing  a  noisy  normal: 

SlxSlySlzl  /I +5/1  Nx  +  5Nx 
S2xS2yS2z  12  +  812  =  Ny  +  8Ny 
_S3x  S3y  [/3  +  8/3J  [//z  +  8Nz_ 

In  matrix  notation,  we  can  write: 

S"’(/-t-5/)  =  N+8N 

The  noisy,  normalized,  normal  is: 

Nx  +  8Nx  Ny  +  8Ny  Nz  +  8Nz 
^ noise  =  \n+8N\  ’  \N+8N]  ’  \N+8N\ 

^ noise  ~  ^^niose’  noise’  ^^noise 

In  the  two  dimensional  case,  the  Nx-Ny  error  area  was  a  quadrilateral.  In  three  dimensions, 
the  Nx-Ny-Nz  error  volume  will  be  an  eight  sided  polyhedra.  In  two  dimensions,  after  nor¬ 
malization,  the  normal  was  projected  onto  the  unit  normal  circle.  In  three  dimensions,  after 
normalization,  the  normal  is  projected  onto  the  unit  normal  sphere. 

We  define  the  orientation  error  to  be  the  angle  between  the  nominal  unit  normal  and  the 
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noisy  unit  normal.  The  orientation  error  depends  on  how  we  define  the  5l  terms.  Initially  we 
defined  the  worst  case  error  to  be: 

5/  =  ±3a. 

I 

By  letting  II,  12,  and  13  independently  take  on  a  worst  case  +3Gi  or  -30;  error,  we  generated 
8  potential  worst  case  errors.  However,  the  probability  of  having  II,  12,  and  13  simulta¬ 
neously  have  3a  values  is  very  small  (approximately  0.013^). 

A  more  realistic  method  of  determining  the  error  is  to  have  the  planner  conduct  a  simulation 
using  the  intensity  noise  function.  If  we  know,  the  mean  intensity  of  each  light  source  (This 
can  be  determined  if  the  object’s  pose  is  known,  if  the  light  source  directions  are  known,  and 
if  the  light  source  radiances  are  known.)  and  the  corresponding  value  of  G;,  we  can  calculate 
a  noisy  surface  orientation  using  the  known  light  source  positions.  At  each  iteration,  a  ran¬ 
dom  intensity  for  each  light  source  is  determined  based  on  the  mean  light  source  intensity 
and  Gj  for  that  light  source  intensity.  These  three  noisy  intensity  values  are  used  to  determine 
a  noisy  surface  orientation.  We  repeat  this  1000  times.  We  calculate  the  mean,  noisy,  surface 
orientation  to  be  the  center  of  mass  of  the  1000  noisy  surface  orientations: 

1000  1000  1000 

I  I  ^^ynoiseh  S 

/V  .  =  ‘  ~  ^ _  '  =  1  <  =  1 

1000  ’  1000  ’  1000 


Using  each  noisy  surface  orientation  and  the  mean,  noisy,  surface  orientation,  we  calculate 
the  orientation  error. 


^err 


•  N  ■  ) 

noise  ‘^'noise> 


Using  the  1000  values  of  orientation  error  that  our  simulation  produced,  we  calculate  the 
standard  deviation  of  the  orientation  error,  G(0en.).  (Orientation  error  is  a  positive  valued 
function.  We  make  it  a  two  sided  function  when  calculating  the  standard  deviation.) 

The  total  orientation  error  for  each  illumination  cover  is  the  sum  of  the  errors  for  each 
face  illuminated  by  the  cover.  Where  the  error  for  each  face  is  3  standard  deviations.  For  n 
faces: 

n 

^,o.alerr=  l3[a(e_)]. 

/  =  1 

We  generated  a  distribution  of  normalized  normals  for  one  case.  The  intensity  noise  function 
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is  from  section  6. 1 . 

Viewpoint  =  (0,  0,  1) 


-0.D5-0.025  0  0.025 

Fig.  14.  3D  Normalized  noisy  normals:  N  =  (0,  0,  1),  SI  =  (-.526,  0.0,  .851),  S2  =  (-.809, 
-.309,  .5),  S3  =  (-.809,  .309,  .5),  II  =  12  =  13  =  150 


5.1.1.  Source  Normalization 


In  any  real  system,  the  light  sources  will  have  different  radiances.  This  will  affect  the  value 
of  a(0err)-  If  the  three  light  sources  used  to  determine  N  do  not  have  the  same  radiances,  the 
three  raw  intensity  values  (II,  12,  and  13)  need  to  be  normalized.  We  normalize  to  the  mini¬ 
mum  light  source  radiance.  If  we  define  minimax  to  be  the  minimum  light  source  radiance 
of  the  three  light  sources  that  we  are  using,  then  for  three  measured  intensities,  our  normal¬ 
ization  is; 


il 


norm 


meas 


.minimax. 


meas 


^  .minimax. 

- )<2„ 

meas 


_  ^minimax^.^ 


where  ilmax  is  the  radiance  of  light  source  one. 

The  normalization  process  affects  the  variance  of  the  measured  intensities  and  therefore 
affects  a(9err)-  If  the  planner  has  knowledge  about  light  source  radiances,  then  it  can  take 
this  information  into  account  when  it  predicts  what  a(0err)  we  can  expect. 

The  planner  will  predict  the  mean  light  source  intensities  for  a  face  based  upon  the  face’s 
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assumed  pose,  and  the  known  light  source  radiances: 

i^plan  ^ 

=  '2.„.(52.iV) 


where  SI  is  the  direction  of  light  source  one,  and  N  is  the  normal  of  the  face. 

These  mean  intensities  should  be  the  intensities  that  irradiate  the  CCD  of  our  camera.  These 
mean  intensities  determine  the  variance  of  our  measurement.  So,  for  each  mean  intensity, 
our  planner  determines  the  variance  (actually  the  standard  deviation)  that  we  expect  to  mea¬ 
sure.  We  will  call  these  three  values  On-,  <15,  . 


However,  the  normalization  of  the  intensity  values  alfects  these  variances.  So  that: 


plan 


=  ( 


minimax^ 
- )a_ 

'1......  il 


^^max  '2 


.minimax^ 

‘’'V  ■ 


The  planner’s  normalized  predicted  intensities  are: 


il 


=  ( 


minimax.  - 


)'l, 


normpian  '  ;I  '"plan 
max 


i2 


=  ( 


minimax 


)'2„ 


normpian  '  /2  '  '^plan 

max 


13 


_  ^ minimax^— - 

normpian  ~  '  ^^plan 

max 


So,  the  intensity  distributions  used  by  the  planner  are: 


A^(il 


normpian'  ilp,, 


normpian' 


normpian' 


The  planner  can  then  use  this  distribution  in  its  simulation.  The  planner  uses  these  distribu- 
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tions  to  produce  1000  noisy  sets  of  intensity  values.  Then,  it  uses  these  1000  noisy  sets  of 
intensity  values  to  calculate  1000  values  of  Nnojse  •  Using  the  1000  values  of  N^oise  >  Ihe  plan¬ 
ner  can  calculate  the  standard  deviation  of  surface  orientation  error,  o(0err)- 

As  can  be  seen,  the  error  prediction  part  of  the  planner  is  becoming  a  simulator.  Generally, 
as  the  planner  incorporates  more  of  the  factors  that  influence  the  measurements,  the  plan¬ 
ner’s  predictions  will  become  more  accurate.  It  is  an  open  question  how  accurate  a  planner 
should  be.  This  will  depend  upon  the  application  and  the  significance  of  the  factors  affecting 
the  measurements.  The  amount  of  error  caused  by  violating  assumptions  depends  on  how 
severely  they  are  violated,  and  on  how  sensitive  the  measurements  are  to  these  violated 
assumptions. 

5.2.  Light  Intensity  Variance 

The  variance  of  the  camera  and  digitizer,  ,  is  a  function  of  light  intensity  as  shown  by 
Healey  [26,  27].  We  have  taken  an  empirical  approach  to  determining  the  value  of  the  func¬ 
tion.  In  general,  our  measurements  are  in  good  agreement  with  Healey’s.  However,  we  do 
observe  deviation  from  Healey’s  data  for  very  small  values  of  I,  and  for  values  of  I  which 
are  greater  than  175.  Healey’s  data  does  not  go  beyond  this  point.  We  are  ignoring  fixed  pat¬ 
tern  noise,  which  is  a  collection  site  to  collection  site  nonuniformity  in  charge  collection  due 
to  processing  limitations,  since  we  are  making  measurements  at  one  pixel.  (If  we  were 
inspecting  a  plane  for  spatial  deviations,  we  would  need  to  consider  fixed  pattern  noise.)  The 
noise  we  are  measuring  is  composed  of  shot  noise,  dark  current  noise,  amplifier  noise  and 
quantization  noise  (amplifier  and  quantization  noise  are  independent  of  I).  At  high  levels  of 
I,  the  poisson  distribution  of  shot  noise  approaches  a  normal  distribution  [36].  We  estimate 
dark  current  noise  with  a  constant. 

5.3.  Camera  Viewpoint  Selection 

We  need  to  position  a  camera  so  that  it  views  all  of  the  faces  illuminated  by  each  illumina¬ 
tion  aspect.  Camera  visibility  is  determined  in  an  identical  manner  to  determining  light 
source  visibility.  The  only  difference  is  that  the  set  of  potential  camera  viewpoints  may  be 
different  from  the  set  of  light  source  directions.  Given  a  set  of  potential  camera  viewpoints, 
virtual  cameras  are  positioned  at  each  viewpoint.  Object  face  visibility  is  determined.  If  the 
set  of  illuminated  faces  is  a  subset  of  the  set  of  visible  faces,  then  the  viewpoint  is  added  to  a 
list  of  candidate  viewpoints.  The  viewpoint,  from  the  list  of  candidates,  that  views  the  set  of 
faces  to  be  inspected  with  the  least  foreshortening  is  selected  as  the  camera  viewpoint.  The 
idea  of  this  metric  is  to  maximize  the  area  of  each  inspected  face  in  our  image.  This  should 
make  defects  maximally  visible.  Formally,  over  the  set  of  possible  viewpoints,  we  try  to 
minimize; 


max{ 


_l _ 1_  1 

COS0|’  00502’  ’  COS0^ 


) 


where  Gj  measures  the  angle  between  the  viewpoint  and  faces  1  ...  n. 
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5.4.  Error  Sources 

There  are  two  basic  types  of  errors  in  photometric  measurements  of  lambertian  objects;  ran¬ 
dom  errors  and  fixed  errors.  Random  errors  are  due  to  aj.  These  are  the  errors  that  we  try  to 
predict  with  our  planner.  Fixed  errors  include:  errors  in  light  source  direction,  errors  in  light 
source  radiance,  errors  in  the  photometric  function  (deviation  from  pure  lambertian).  Fixed 
errors  can  be  accounted  for  by  a  careful  calibration  procedure.  [28] 

There  is  also  a  third  type  of  error.  This  is  the  potential  error  in  object  pose.  Our  planner 
assumes  that  the  object  being  viewed  is  in  a  certain  orientation.  If  the  object  is  in  a  different 
orientation,  the  planner’s  predicted  surface  orientation  error  will  be  erroneous.  A  change  in 
orientation  will  cause  a  change  in  the  incident  angles  between  the  light  sources  and  object 
faces.  This  will  cause  a  change  in  expected  mean  light  source  intensities,  Gj  ,  and  thus  the 
standard  deviation  of  surface  orientation  error,  G(0err). 

The  amount  of  change  in  mean  light  source  intensities  depends  on  the  lambertian  photomet¬ 
ric  function  (the  cosine  function).  If  the  incident  angle  is  near  normal,  a  small  change  in 
rotation  will  not  cause  a  large  change  in  the  expected  mean  light  source  intensity.  If  the  inci¬ 
dent  angle  is  small,  a  small  change  in  rotation  could  cause  a  large  change  in  the  expected 
mean  light  source  intensity. 

5.5.  Texture  and  Cj 

Our  use  of  Gj ,  assumes  that  the  planar  surfaces  of  our  objects  have  no  texture.  Any  spatial 
variation  of  the  surface  due  to  texture  will  cause  an  increase  in  the  measured  value  of  G;  , 
beyond  what  is  caused  by  the  CCD  alone.  It  would  be  possible  to  form  an  aggregate  G^j  . 

cP:  =  +a? 

^CCD  ^texture 

The  aggregate  would  be  composed  of  two  terms.  One  term  would  be  the  intensity  variation 
caused  by  the  camera  and  digitizer.  The  second  term  would  be  the  intensity  variation  due  to 
texture.  (These  terms  are  independent.)  In  this  way,  surface  defects  larger  than  the  surface 
deviation  caused  by  texture  could  be  reliably  detected. 
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6.  3D  Convex  Lambertian  Implementation 


The  planner  was  implemented  in  Lucid™  Common  Lisp.  Experiments  were  carried  out 
using  a  Puma  560  robot  and  an  array  of  light  sources  placed  at  the  vertices  of  a  1 -frequency 
icosahedron.  (For  structural  reasons,  61  faces  actually  exist  and  36  of  the  vertices  have  light 
sources).  A  Sony  XC-57  camera,  with  a  Nikon  AF  Nikkor  Micro  105mm  F2.8  lens,  was 
mounted  on  the  Puma’s  end  effector.  The  Puma  Robot,  with  the  camera  mounted  on  it  could 


only  reach  22  of  the  61  icosahedron  faces  (Camera  viewpoints  were  located  at  the  faces  of 
the  1-frequency  icosahedron.)  because  of  robot  workspace  constraints.  Images  were  digi¬ 
tized  using  an  Androx  ICS-400  digitizer. 


Robot 


CCD 

Camera 


Light 

Source 

Array 


Object 

Unaer 

Inspection 


Fig.  15.  Experimental  Setup 


6.1.  Measurement  of  Light  Intensity  Variance 

In  order  to  measure  Oj,  illumination,  viewing  geometry,  camera  temperature,  and  CCD  volt¬ 
ages  must  be  controlled.  Any  variance  in  any  of  these  parameters  will  cause  added  variation 
to  Oj.  We  used  a  light  source  controlled  with  a  linear  DC  power  supply  and  used  a  linear  DC 
power  supply  to  power  the  Sony  XC-57  camera  (We  found  that  controlling  the  camera  with 
a  standard  switching  camera  power  supply  increased  Oj.)  Camera  temperature  was  held  con¬ 
stant  by  allowing  the  camera  to  reach  steady  state  temperature  with  respect  to  its  environ¬ 
ment.  Viewing  geometry  was  maintained  by  rigidly  supporting  the  camera  and  target.  Our 
target  was  of  uniform  matte  reflectance. 

We  selected  four  pixels  on  the  CCD,  and  took  1000  images  of  our  fixed  scene.  Illumination 
was  changed  by  varying  the  incident  angle  between  the  light  source  and  target.  We  took 
measurements  between  mean  intensities  of  8  (dark  current  value)  and  227.  Normality  was 
checked  by  plotting  a  histogram  of  intensity  for  each  pixel,  and  by  using  the  Kolmogrov- 
Smirnov  test  [29].  The  average  significance  level  of  the  Kolmogrov-Smirnov  test  was  0.342. 


A  representative  histogram  is  shown  below: 


For  each  pixel  we  calculated  the  mean,  variance,  and  standard  deviation  of  the  intensity 
measurements.  We  fitted  a  second  degree  polynomial  to  the  raw  data,  and  used  this  polyno¬ 
mial  in  our  planner.  The  plot  of  the  standard  deviation  of  intensity  is  shown  below.  The  fitted 
polynomial  is  shown  as  the  solid  line. 


Intensity 

Fig.  17.  Plot  of  Gj 
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7.  3D  Convex  Lambertian  Experiments 

Experiments  were  conducted  to  validate  the  lambertian  illumination  planner.  We  wanted  to 
verify  that  the  illumination  plans  generated  by  the  planner  were  valid  and  accurate.  In  order 
to  do  this,  we  tested  the  results  of  the  planner  for  a  set  of  illumination  plans.  One  set  of  plans 
tested  was  near  the  most  accurate  illumination  plan  generated  by  our  illumination  planner. 
The  other  set  of  plans  tested  was  near  the  least  accurate  illumination  plan  generated  by  our 
illumination  planner.  The  results  of  the  two  plans  were  compared  with  each  other  and  with 
the  planner’s  predictions.  We  also  verified  that  the  light  source  viewpoint  visibility  and  cam¬ 
era  viewpoint  visibility  predicted  by  the  planner  were  correct. 

7.1.  Chalk  Cube 

We  milled  a  cube  out  of  ‘railroad  chalk’.  The  cube  was  mounted  under  our  array  of  light 
sources.  The  cube  was  oriented  so  that  the  X,  Y,  and  Z  axes  intersect  at  the  center  of  the 
cube.  The  X  axis  intersects  one  edge  of  the  cube.  The  Y  axis  intersects  another  edge  of  the 
cube.  The  Z  axis  intersects  the  center  of  the  top  face  of  the  cube: 


Fig.  18.  Geometric  Model  of  Chalk  Cube 
The  nominal  orientation  of  the  cube’s  faces  is  shown  in  the  table  below. 


Face 

Nx 

Ny 

Nz 

1 

-0.707 

0.707 

0.0 

2 

-0.707 

-0.707 

0.0 

3 

0.707 

-0.707 

0.0 

4 

0.707 

0.707 

0.0 

5 

0.0 

0.0 

1.0 

6 

0.0 

0.0 

-1.0 
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The  possible  light  source  directions: 


Light 

Source 

Nx 

Ny 

Nz 

1 

-0.3090 

-0.5000 

0.8090 

2 

0.3090 

-0.5000 

0.8090 

3 

0.0000 

0.0000 

1.0000 

4 

0.0000 

-0.8507 

0.5257 

5 

0.5257 

0.0000 

0.8507 

6 

-0.5257 

0.0000 

0.8507 

7 

-0.8090 

-0.3090 

0.5000 

8 

-0.5000 

-0.8090 

0.3090 

9 

0.5000 

-0.8090 

0.3090 

10 

0.8090 

-0.3090 

0.5000 

11 

0.3090 

0.5000 

0.8090 

12 

-0.3090 

0.5000 

0.8090 

13 

-1.0000 

0.0000 

14 

0.3090 

0.5000 

15 

0.3090 

0.5000 

16 

-0.5257 

0.0000 

17 

-0.5257 

0.0000 

18 

0.8507 

0.5257 

19 

0.0000 

0.0000 

20 

-0.8090 

-0.3090 

21 

-0.8090 

-0.3090 

22 

0.0000 

0.0000 

23 

0.8090 

0.3090 

24 

0.8090 

0.3090 

25 

-0.8507 

-0.5257 

26 

0.5257 

0.0000 

27 

-0.8507 

0.5257 

0.0000 

28 

-0.8090 

-0.3090 

-0.5000 

29 

0.8090 

-0.3090 

-0.5000 

30 

0.0000 

1.0000 

0.0000 

31 

-0.3090 

-0.5000 

-0.8090 

32 

0.3090 

-0.5000 

-0.8090 

33 

0.8090 

0.3090 

-0.5000 

34 

0.5000 

0.8090 

-0.3090 

35 

-0.5000 

0.8090 

-0.3090 

36 

-0.8090 

0.3090 

-0.5000 
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The  possible  viewing  directions; 


View 

Nx 

Ny 

Nz 

5 

-0.3090 

-0.1667 

0.9363 

15 

-0.1667 

0.9363 

0.3090 

16 

0.1667 

0.9363 

0.3090 

17 

0.0000 

0.3568 

0.9342 

18 

0.0000 

0.6667 

0.7454 

19 

-0.3090 

0.1667 

0.9363 

21 

-0.5774 

0.5774 

0.5774 

22 

-0.2697 

0.7697 

0.5787 

23 

-0.5787 

0.2697 

0.7697 

24 

-0.7697 

0.5787 

0.2697 

25 

0.5774 

0.5774 

0.5774 

26 

0.2697 

0.7697 

0.5787 

27 

0.5787 

0.2697 

0.7697 

28 

0.7697 

0.5787 

0.2697 

30 

-0.7454 

0.0000 

0.6667 

34 

0.7454 

0.0000 

0.6667 

37 

-0.3568 

0.9341 

0.0000 

41 

0.3568 

0.9341 

0.0000 

The  best  case  inspection  plans,  assuming  ideal,  equal  radiance,  light  sources,  found  by  our 
planner  are  below.  Cover  1  and  cover  2  are  not  implementable  because  the  indicated  aspects 
are  not  viewable. 


Cover 

Aspect 

Faces 

Covered 

View 

Light  Sources 

1 

1 

2,3 

none 

13,31,32 

1 

2 

1,4,5 

18 

11,23,24 

2 

1 

1,4 

17 

30,  34,  35 

2 

2 

2,  3,5 

none 

2,8,9 

3 

1 

1,2 

19 

16,  27,  36 

3 

2 

3,4,5 

30 

5,  10,  14 

4 

1 

3,4 

34 

17,  26,  29 

4 

2 

1,2,5 

30 

6,7,  15 

Inspectable  faces  are  defined  as  the  object  faces  that  can  be  viewed  from  the  set  of  possible 
viewpoints  (face  6  is  not  inspectable).  For  each  light  source,  we  determine  which  of  the 
inspectable  faces  are  illuminated.  (If  a  non-inspectable  face  is  illuminated,  we  treat  it  as  a 
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“don’t  care”.)  The  different  combinations  of  illuminated  faces  form  illumination  aspects. 
These  aspects  are  formed  into  exact  illumination  covers.  We  used  our  “largest  aspect”  heu¬ 
ristic,  with  a  search  depth  of  one,  to  form  the  exact  covers.  It  is  possible  that  an  illumination 
aspect  contains  a  combination  of  object  faces  that  is  not  viewable.  This  leads  to  the  cases 
like  covers  1  and  2,  which  both  contain  illumination  aspects  that  are  not  viewable.  (If  a 
potential  viewpoint  views  a  non-inspectable  face,  we  treat  it  as  a  “don’t  care”.) 

We  implemented  a  near  best  case  inspection  plan  for  faces  3  and  4  using  light  sources  22, 
29,  33  and  viewpoint  34.  The  near  worst  case  plan  for  faces  3  and  4  was  light  sources  22,  26, 
29  and  viewpoint  34.  The  planner  determined  that  faces  1,  2,  and  5  could  only  be  illumi¬ 
nated  by  light  sources  6, 7, 15.  The  camera  viewpoint  for  faces  1,  2,  and  5  was  viewpoint  30. 

We  implemented  these  inspection  plans  on  our  experimental  setup.  The  results,  using  mea¬ 
sured  radiance  light  sources: 


Planner 

Predictions 

Measurements 

Face 

Light 

Sources 

Pixel 

Number 

<y(0err)  ' 
Degrees 

(^(Qeix)  - 
Degrees 

3 

22,29,33 

1 

1.92 

1.81 

3 

22,29,33 

2 

1.92 

1.81 

3 

22,26,29 

1 

2.55 

2.90 

3 

22,26,29 

2 

2.55 

3.00 

4 

22,29,33 

3 

1.95 

1.98 

4 

22,29,33 

4 

1.95 

2.10 

4 

22,26,29 

3 

2.78 

3.01 

4 

22,26,29 

4 

2.78 

3.06 

1 

6,7,15 

5 

1.82 

1.91 

1 

6,7,15 

6 

1.82 

1.85 

2 

6,7,15 

7 

1.75 

2.14 

2 

6,7,15 

8 

1.75 

2.08 

5 

6,7,15 

9 

1.80 

1.70 

5 

6,7,15 

10 

1.80 

1.98 

5 

6,7,15 

11 

1.80 

1.72 

5 

6,7,15 

12 

1.80 

1.68 

Measurements  were  made  by  taking  1000  images  with  each  light  source.  A  small  number  of 
pixels  (4)  were  recorded  from  each  image.  This  produced  a  data  stream  for  that  pixel.  By 
combining  data  streams  for  3  light  sources,  we  calculated  a  mean  surface  orientation  and  the 
standard  deviation  of  surface  orientation  error. 

In  general,  the  measured  standard  deviation  of  orientation  error  are  within  20%  of  the  plan¬ 
ner.  Many  are  within  10%. 
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The  difference  between  the  near  best  case  and  near  worst  case  plans  should  be  noted.  For 
Face  3,  the  near  best  case  illumination  plan  results  in  a  5.76°  predicted  error,  3G(0err)-  In 
contrast,  the  near  worst  case  illumination  plan  results  in  7.65°  predicted  error.  For  Face  4, 
the  near  best  case  illumination  plan  results  in  a  5.85°  predicted  error.  In  contrast,  the  near 
worst  case  illumination  plan  results  in  8.34°  predicted  error. 

Images  of  Faces  3  and  4  using  light  sources  22, 29,  and  33.  (Face  5  is  not  illuminated.) 


Fig.  19.  Intensity  Images:  Faces  3  and  4  of  cube 


Needle  map  produced  by  the  intensity  images  from  light  sources  22,  29,  and  33. 
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Fig.  20.  Needle  Map:  Faces  3  and  4  of  cube 
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Fig.  21.  Intensity  Images:  Faces  6,  7,  and  15  of  cube 


Needle  map  produced  by  the  intensity  images  from  light  sources  6,  7,  and  15. 


Fig.  22.  Needle  Map  Faces:  6,  7,  and  15  of  cube 


Illumination  aspects  from  the  1,1,1  viewing  direction  (Numbers  are  light  source  numbers. 


Light  sources  that  are  shaded  the  same  belong  to  the  same  illumination  aspect.): 


Illumination  aspects  from  the  -1, 1,  1  viewing  direction  (Numbers  are  light  source  numbers. 
Light  sources  that  are  shaded  the  same  belong  to  the  same  illumination  aspect.); 


Fig.  24.  Illumination  aspects  from  the  -1,  1,  1  viewing  direction 


Illumination  aspects  from  the  0,  0,  1  viewing  direction  (Numbers  are  light  source  numbers. 
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Light  sources  that  are  shaded  the  same  belong  to  the  same  illumination  aspect.): 


Fig.  25.  Illumination  aspects  from  the  0,  0,  1  viewing  direction 


page  37 


8.  2D  Concave  Lambertian  Illumination 

Now  we  consider  the  illumination  of  simple  2D  concavities,  concavities  composed  of  two 
surface  edges  that  are  fully  visible  to  each  other.  The  illumination  of  a  concavity  requires  the 
determination  of  visibility  regions  for  the  potential  light  sources.  For  the  simple  concavities 
that  we  are  considering  in  our  work,  we  define  the  visibility  region  as  the  extension  of  the 
surface  edges  that  form  the  concavity.  For  example,  if  the  concavity  has  sides  at  45°  and 
135°,  the  region  that  can  see  the  concavity  is  the  visibility  region  extending  from  45°  to 
135°.  Any  viewpoint  in  the  visibility  region  can  see  both  faces  of  the  concavity,  so  the  exact 
cover  problem  is  trivial.  (We  do  not  consider  partial  visibility  of  a  concave  edge.) 

1 35°  Visibility  Region  4-^0 

\r  V 


Fig.  26.  Concave  Visibility  Region 

Once  we  have  determined  the  light  source  visibility  region,  we  need  to  determine  the  opti¬ 
mal  location  for  light  source  placement  within  that  region. 

Nayar  [31]  showed  that  a  shape  from  intensity  method,  such  as  photometric  stereo,  applied 
to  a  concavity  would  produce  a  “pseudo  shape”.  The  pseudo  shape  is  shallower  than  the  real 
concavity  and  the  pseudo  shape  is  invariant  to  light  source  placement.  So,  no  matter  where 
we  place  our  light  sources  within  the  visibility  region,  the  pseudo  shape  recovered  using  a 
method  such  as  photometric  stereo  will  produce  the  same  shape. 

So,  if  the  pseudo  shape  is  invariant  to  light  source  placement,  what  constitutes  the  light 
source  placement  problem.  We  define  two  possible  concave  light  source  placement  prob¬ 
lems.  The  first  problem  is  inspection  of  the  pseudo  shape.  Deviations  in  the  real  shape  will 
cause  deviations  in  the  pseudo  shape.  Therefore,  we  can  detect  deviations  in  the  real  shape 
by  comparing  the  ideal  pseudo  shape  with  the  measured  pseudo  shape.  Different  light  source 
placements  will  give  different  amounts  of  uncertainty  in  the  pseudo  shape.  We  want  to  find 
the  light  source  placement  that  yields  the  highest  reliability.  The  second  concave  inspection 
problem  involves  using  the  pseudo  shape  to  recover  the  real  shape  using  Nayar’s  interactive 
shape  recovery  algorithm.  Depending  on  the  initial  reliability  of  the  pseudo  shape,  the  final 
shape  of  the  iterative  algorithm  may  be  closer  to  the  actual  shape.  One  would  want  to  find 
the  light  source  placement  that  produced  the  most  accurate  final  shape.  In  this  paper,  we  will 
discuss  the  first  problem,  inspection  of  the  pseudo  shape. 

The  goal  of  inspecting  the  pseudo  shape  is  to  find  the  light  source  positions  that  produce  the 
minimum  amount  of  uncertainty  in  the  pseudo  shape.  The  way  to  think  about  this  problem  is 
that  interreflection,  causes  a  distorted  shape,  the  pseudo  shape.  The  pseudo  shape  stays  the 
same  no  matter  where  we  place  our  light  sources.  So,  the  apparent  object  we  are  inspecting 
stays  the  same  no  matter  where  we  place  our  light  sources.  This  is  equivalent  to  inspecting  a 
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non-interreflecting  lambertian  object  whose  shape  is  equivalent  to  the  pseudo  shape.  The 
process  of  interreflection  that  creates  the  pseudo  shape  is  global,  encompassing  all  of  the 
surfaces  that  form  the  concavity.  However,  the  problem  of  inspecting  the  pseudo  shape  is 
local.  The  reliability  at  each  point  of  the  pseudo  shape  only  depends  on  the  intensity  at  that 
point,  not  on  any  other  points  of  the  pseudo  shape. 

We  can  use  the  forward  graphics  solution  [32]  to  predict  the  brightness  of  the  concavity.  The 
basic  interreflection  equation  for  diffuse  surfaces  is: 

Radiosity  =  Emission  +  Interreflection 

B{u)  =  E{u)  +  p{u)  {u,v)  B  {v)  dv 
D 

F  called  the  form  factor.  It  is  the  fraction  of  the  energy  leaving  surface  u  that  arrives  at  sur¬ 
face  V.  E  is  the  surface  radiance  due  to  a  tight  source,  p  is  the  albedo  of  the  surface.  B  is  the 
aggregate  surface  radiance.  D  is  all  of  the  surfaces  in  the  environment. 

In  discrete  form, 

n 

Bj  =  Ej  +  PjJ^F(i,j)B.  j=l,n 

i=  I 

where  n  is  the  number  of  elements  in  the  environment. 

For  the  two-dimensional  case,  with  elements  i  and  j.  r  is  the  distance  between  the  centers  of 
the  elements.  Nj  is  the  normal  of  element  i.  Nj  is  the  normal  of  element]. 


Fig.  27.  Interreflection  Geometry 


The  form  factor  [33]  between  elements  i  and]  is: 

COS0.COS0. 

FUJ)  =  ViiJ) - ^;—^ds 

V  is  the  visibility  between  face  i  and  face].  V  =  1  if  face  i  can  see  face].  V  =  0  otherwise. 
We  use  point  to  point  form  factors  calculated  between  the  center  of  each  element,  with  a  uni¬ 
form  mesh  and  constant  elements. 

We  can  express  the  discrete  form  of  the  radiosity  equation  in  matrix  form 


U-PF)B  =  E 
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E  and  B  are  vectors  of  length  n.  E  is  the  radiance  of  each  element  due  to  the  light  source.  B 
is  the  aggregate  radiance  of  an  element  due  to  interreflection  and  the  light  source. 

F  is  the  square  matrix  of  form  factors. 


^11 

Fi2  ... 

^21 

F22  ■■■ 

■  ^33 

P  is  the  diagonal  albedo  matrix. 

P,  0  0  0 

0  P,  0  0 

P  =  2 

0  0  P3  0 

0  0  0  P„ 


In  the  forward  solution,  we  know  P,  F,  and  E.  We  want  to  solve  for  B,  the  brightness  of  the 
concavity  with  interreflection.  The  matrix  equation  is  solved  using  the  successive  over 
relaxation  method  [34],  with  w  =  1.4.  Convergence  usually  occurs  in  approximately  15  iter¬ 
ations. 

We  solve  the  forward  problem  for  two  light  sources,  S 1  and  S2.  S 1  produces  the  brightness 
distribution  Bl.  S2  produces  the  brightness  distribution  B2.  Once  we  have  B1  and  B2,  we 
can  solve  for  the  pseudo  shape  using  Bl,  SI,  B2,  and  S2. 


Six  Sly 

-1 

~Bl  [/f 

Nx[i] 

S2x  Sly 

_Nyli]_ 

Where  Nx  and  Ny  are  the  normals  of  the  pseudo  shape,  and  n  is  the  number  of  face  ele¬ 
ments. 

If  our  goal  is  to  inspect  the  pseudo  shape  with  the  most  reliability  possible,  then  the  analysis 
from  here  on  is  very  similar  to  the  2D  and  3D  convex  cases.  The  difference  between  the  con¬ 
vex  case  and  the  pseudo  shape  case  is  that  the  pseudo  shape  is  non-planar.  The  uncertainty 
for  a  planar  convex  face  is  constant  since  the  variance  of  the  face  depends  on  the  face’s  nor¬ 
mal  direction  and  the  light  source  direction.  A  pseudo  shape  face  is  curved.  Therefore,  the 
uncertainty  along  the  pseudo  shape  face  varies.  We  could  minimize  the  average  angular  ori¬ 
entation  error  of  a  pseudo  shape  face,  or  if  we  were  interested  in  a  particular  point,  we  could 
minimize  the  angular  orientation  error  of  that  point.  For  the  experiments  that  follow,  we 
seek  to  minimize  the  error  at  the  center  of  each  face. 

If  B 1  and  B2  are  the  nominal  light  source  intensities,  noise  in  either  B 1  or  B2  will  cause  an 
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error  in  Nx  and  Ny,  producing  a  noisy  normal: 


Six 

-1 

Bl[fi 

+  5B1  [j] 

A^jc[j]  +5Nx[i] 

S2x  Sly 

B2  [(■] 

+  8fi2[/] 

Ny  [i]  +  5Ny  [i] 

If  we  were  to  measure  B 1  and  B2,  both  of  which  might  be  corrupted  by  errors,  and  we  were 
to  solve  for  Nx  and  Ny,  we  would  normalize  the  resulting  values  of  Nx  and  Ny  because  the 
surface  normal  is  by  definition  a  unit  vector.  The  noisy,  normalized,  normal  is: 

_ 

-  |^[,]  +8Ar[/]|’  |Af[,]  +5iV[/]| 

I  —  i,  n 

N[i]  .  =  Nx[i]  .  ,Ny[i]  . 

‘  noise  ^  ^  niose  ^  noise 


We  define  the  orientation  error  to  be  the  angle  between  the  nominal  unit  normal  and  the 
noisy  unit  normal.  We  have  the  planner  conduct  a  simulation  using  the  intensity  noise  func¬ 
tion.  If  we  know,  the  nominal  intensity  of  each  light  source  (This  can  be  determined  if  the 
object’s  pose  is  known,  if  the  light  source  directions  are  known,  and  if  the  light  source’s 
albedos  are  known.)  and  the  corresponding  value  of  ctj,  we  can  calculate  a  noisy  surface  ori¬ 
entation  using  the  known  light  source  positions.  At  each  iteration,  a  random  intensity  for 
each  light  source  is  determined  based  on  the  mean  light  source  intensity  and  Cj  for  that  light 
source  intensity.  These  three  noisy  intensity  values  are  used  to  determine  a  noisy  surface  ori¬ 
entation.  We  repeat  this  1000  times.  We  calculate  the  mean,  noisy,  surface  orientation  to  be 
the  center  of  mass  of  the  1000  noisy  surface  orientations: 

1000  1000 

y  [Nx[i]  .  ]  y  [A^y[/]  .  ]  i  =  i  n 

^  ^  noise^  j  L  ^  L  i  noise^  j  ^  ...^n 

Using  each  noisy  surface  orientation  and  the  mean,  noisy,  surface  orientation,  we  calculate 
the  orientation  error. 


6[(]  =  acos  (Af  [«■]  .  )  /  -  /  « 

Using  the  1000  values  of  orientation  error  that  our  simulation  produced,  we  then  determine 
the  standard  deviation  of  surface  orientation  error,  CT{0en.). 

We  have  performed  simulations  on  a  range  of  simple  2D  concavities  to  explore  the  relation¬ 
ship  between  a(0err)  and  concavity  shape. 
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The  first  concavity  we  looked  at  had  a  concavity  of  140°.  p=  0.8.  n  =  500.  The  intensity  of 
both  light  sources  was  200.  Both  sides  of  the  concavity  are  the  same  length. 


Face  “A" 


Face  “B’ 


20°  . 


> 


. . 


Fig.  28.  140  2D  Concavity 

The  shape  and  pseudo  shape  are  shown  here  (The  pseudo  shape  is  the  more  concave  shape.): 


Fig.  29.  140°  2D  Concavity  Shape  and  Pseudo  Shape 


The  error  surface  of  Ogn.  f^ce  for  Face  “A”  is  (SI,  S2,  and  error  are  all  in  degrees): 


3a(0en.)  for  center  of  Face  A 


/  S2  kor 


3a(6en.)  for  center  of  Face  A 


isaso 


3a(0err)  for  Center  of  Face  A 


150  50 


Fig.  30.  140°  2D  Concavity  Error  Surface  for  center  of  Face  A 


The  maximum  error  is  37.5°.  The  minium  error  is  1.5°.  The  search  was  conducted  in  5 
increments. 


page  42 


The  second  concavity  we  looked  at  had  a  concavity  of  90°.  p=  0.8.  n  =  500.  The  intensity  of 
both  light  sources  was  200.  Both  sides  of  the  concavity  are  the  same  length. 


Fig.  31.  90°  2D  Concavity 

The  shape  and  pseudo  shape  are  shown  here  (The  pseudo  shape  is  the  more  concave  shape.): 


Fig.  32.  90°  2D  Concavity  Shape  and  Pseudo  Shape 


The  maximum  error  is  35.6°.  The  minium  error  is  2.0°.  The  search  was  conducted  in  5° 
increments. 
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The  third  concavity  we  looked  at  had  a  concavity  of  45°.  p=  0.8.  n  =  500.  The  intensity  of 
both  light  sources  was  200.  Both  sides  of  the  concavity  are  the  same  length. 


67.5°  \  /67.5° 


Fig.  34.  45°  2D  Concavity 


The  shape  and  pseudo  shape  are  shown  here  (The  pseudo  shape  is  the  more  concave  shape.): 


Fig.  35.  45°  2D  Concavity  Shape  and  Pseudo  Shape 

The  error  surface  of  eerr_face  for  Face  “A”  is  (SI,  S2,  and  error  are  all  in  degrees): 

3a(0err)  for  Face  center  of  A  3a(eerr)  for  center  of  Face  A 


Fig.  36.  45°  2D  Concavity  Error  Surface  for  center  of  Face  A 


The  maximum  error  is  36.8°.  The  minium  error  is  6.7°.  The  search  was  conducted  in  5° 
increments. 
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As  the  concavity  becomes  more  acute,  the  pseudo  shape  becomes  deeper  and  more  concave. 
In  addition,  as  the  concavity  becomes  more  acute,  the  visibility  region  becomes  more  acute. 
These  two  factors  combine  to  increase  the  minimum  error  for  a  face.  (The  maximum  error  is 
largely  a  function  of  the  separation  between  the  light  sources.) 

As  shown  in  section  3.4.1.,  a  lambertian  shape  will  produce  high  reliability  when  the  angle 
between  the  light  sources  and  normal  are  large.  As  a  concavity  becomes  more  acute,  and  the 
pseudo  shape  becomes  deeper  and  more  concave,  the  maximum  angle  between  viewpoints 
in  the  visibility  region  and  points  along  the  pseudo  shape  will  be  smaller.  This  will  cause  the 
error  to  be  larger,  and  more  of  the  visibility  region  will  produce  large  errors.  For  a  shallow 
concavity,  the  pseudo  shape  will  be  less  concave.  Less  of  the  visibility  region  will  be  at 
acute  angles  to  the  pseudo  shape.  The  maximum  angle  between  viewpoints  in  the  visibility 
region  and  points  along  the  pseudo  shape  will  be  larger.  This  will  cause  the  error  to  be 
smaller,  and  less  of  the  visibility  region  will  produce  large  errors. 

We  also  looked  at  the  variation  of  error  along  each  face  of  the  pseudo  shape.  (Points  0  to  249 
belong  to  pseudo  shape  face  A.  Points  250  to  499  belong  to  pseudo  shape  face  B.  si  =  72.5°, 
s2  =  107.5°,  rho  =  0.8) 


0.0  100.0  200.0  300.0  400.0  500.0  Point  Number 
Fig.  37.  Error  across  face  A  and  Face  B  forl40°  concavity 


0.0  100.0  200.0  300.0  400.0  500.0  Point  Number 
Fig.  38.  Error  across  face  A  and  Face  B  for  90°  concavity 
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0.0  100.0  200.0  300.0  400.0  500.0  Point  Number 
Fig.  39.  Error  across  face  A  and  Face  B  for  45°  concavity 

The  variation  in  error  along  a  face  is  a  function  of  the  curvature  of  the  pseudo  shape  face.  If 
the  pseudo  shape  face  is  approximately  flat,  the  angle  between  the  light  sources  and  face  are 
approximately  constant.  Therefore,  the  error  along  the  face  is  approximately  constant.  If  the 
pseudo  shape  face  has  a  high  curvature,  the  angle  between  the  light  sources  and  face  will 
vary.  This  will  cause  the  error  along  the  face  to  vary.  The  45°  concavity’s  pseudo  shape  has 
the  highest  curvature.  Therefore,  the  error  along  its  face  varies  the  most. 
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9.  3D  Concave  Lambertian  Illumination 

Now  we  consider  the  illumination  of  simple  3D  concavities,  concavities  composed  of  two 
rectangular  faces  that  are  fully  visible  to  each  other.  The  solution  of  the  3D  concave  illumi¬ 
nation  problem  is  very  similar  to  the  solution  of  the  2D  concave  illumination  problem. 

For  the  three-dimensional  case,  with  elements  i  and  j.  r  is  the  distance  between  the  centers  of 
the  elements.  Nj  is  the  normal  of  element  i.  Nj  is  the  normal  of  element  j. 


Fig.  40.  Interreflection  Geometry 


The  form  factor  [35]  between  elements  i  and]  is: 

COS0.COS0. 

=  VUJ) - 4—^44. 

nr 

V  is  the  visibility  between  face  i  and  face].  V  =  1  if  face  i  can  see  face  j.  V  =  0  otherwise. 
dAj  is  the  differential  area  of  facet]. 

We  solve  the  forward  problem  for  three  light  sources,  SI,  S2,  and  S3.  SI  produces  the 
brightness  distribution  Bl.  S2  produces  the  brightness  distribution  B2.  S3  produces  the 
brightness  distribution  B3.  Once  we  have  Bl,  B2,  and  B3,  we  can  solve  for  the  pseudo  shape 
using  Bl,  SI,  B2,  S2,  B3,  and  S3. 

-1 

51z  Bl  Nx 
Six  Sly  Slz  Bl  =  Ny 
_S3x  S3y  53^  [esj  [piz 

Our  goal  is  to  inspect  the  pseudo  shape  with  the  highest  reliability  possible.  The  analysis 
from  here  on  is  identical  to  the  3D  convex  case.  Section  5.1..  We  would  use  Bl,  B2,  B3,  Nx, 
Ny,  and  Nz  to  calculate  the  angular  orientation  error  at  each  point  on  the  pseudo  shape.  The 
difference  between  the  convex  case  and  the  pseudo  shape  case  is  that  the  pseudo  shape  is 
non-planar.  The  uncertainty  for  a  planar  convex  face  is  constant  since  the  variance  of  the 
face  depends  on  the  face’s  normal  direction  and  the  light  source  direction.  A  pseudo  shape 
face  is  curved.  Therefore,  the  uncertainty  along  the  pseudo  shape  face  varies.  We  could  min¬ 
imize  the  average  angular  orientation  error  of  a  pseudo  shape  face,  or  if  we  were  interested 
in  a  particular  point,  we  could  minimize  the  angular  orientation  error  of  that  point.  For  the 
experiments  that  follow,  we  seek  to  minimize  the  error  at  the  center  of  each  face. 
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9.1.  Implementation 

We  implemented  our  3D  concave  planner  in  C  instead  of  in  LISP  because  of  the  large  com¬ 
putational  expense  involved  in  the  forward  prediction  problem.  Our  3D  planner  accepts  the 
vertex  coordinates  of  two  rectangular  faces.  Each  face  is  tessellated  into  an  NxN  grid  of  ele¬ 
ments.  (For  these  experiments  N  =  15.)  All  combinations  of  light  sources  that  are  visible  to 
both  faces  are  included  in  a  potential  source  list.  The  forward  prediction  is  performed  for 
each  combination  of  light  sources  in  the  source  list.  Then,  the  angular  orientation  error  is 
calculated  at  the  center  of  both  faces.  A  viewpoint  that  satisfies  the  minimum  foreshortening 
criteria  is  used. 
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10.  3D  Concave  Lambertian  Illumination  Experiments 

Experiments  were  conducted  to  validate  the  concave  lambertian  illumination  planner.  We 
wanted  to  verify  that  the  illumination  plans  generated  by  the  planner  were  valid  and  accu¬ 
rate.  In  order  to  do  this,  we  tested  the  results  of  the  planner  for  a  set  of  illumination  plans. 
One  set  of  plans  tested  was  near  the  most  accurate  illumination  plan  generated  by  our  illumi¬ 
nation  planner.  The  other  set  of  plans  tested  was  near  the  least  accurate  illumination  plan 
generated  by  our  illumination  planner.  The  results  of  the  two  plans  were  compared  with 
each  other  and  with  the  planner’s  predictions.  We  also  verified  that  the  light  source  view¬ 
point  visibility  and  camera  viewpoint  visibility  predicted  by  the  planner  were  correct. 

10.1.  Chalk  Concavity 

We  constructed  a  simple  concavity.  Two  rectangular  faces  were  milled  out  of  railroad  chalk. 
The  two  faces  were  held  in  a  holder  that  formed  a  90°  concavity. 

The  concavity  was  oriented  so  that  the  X,  Y,  and  Z  axes  intersect  at  the  center  of  the  concav¬ 
ity.  The  X  axis  coincides  with  the  cusp  of  the  concavity.  The  Z  axis  bisects  the  concavity  (it 
points  up).  By  using  a  Macbeth  ColorChecker  color  rendition  chart,  we  determined  that  the 
albedo  of  the  railroad  chalk  was  0.936. 


The  nominal  orientation  of  the  concavity’s  faces  is  shown  in  the  table  below. 


Face 

Nx 

Ny 

Nz 

A 

0.0 

0.707 

0.707 

B 

0.0 

-0.707 

0.707 

Our  light  sources  are  2”  in  diameter  and  they  are  9”  from  our  object.  They  subtend  an  angle 
of  12.5°.  In  order  to  ensure  that  all  of  the  light  source  is  visible  to  each  illuminated  face,  we 
impose  a  limit  on  the  light  source  incident  angle.  The  angle  between  a  light  source  and  Face 
A  or  Face  B  cannot  be  more  than  77.5°.  Seven  light  sources  satisfy  this  condition:  SI,  S2, 
S3,  S5,  S6,  Sll,  and  S12.  We  measured  the  intensities  of  these  light  sources  using  the  same 
material  that  the  chalk  faces  were  made  of. 
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The  possible  light  source  directions: 


Light 

Source 

Nx 

Ny 

Nz 

Source  Intensity 

1 

-0.3090 

-0.5000 

0.8090 

176.2 

2 

0.3090 

-0.5000 

0.8090 

207.0 

3 

0.0000 

0.0000 

1.0000 

207.4 

5 

0.5257 

0.0000 

0.8507 

185.7 

6 

-0.5257 

0.0000 

0.8507 

205.2 

11 

0.3090 

0.5000 

0.8090 

192.3 

12 

-0.3090 

0.5000 

0.8090 

208.2 

The  possible  viewing  directions: 


View 

Nx 

Ny 

Nz 

5 

-0.3090 

-0.1667 

0.9363 

15 

-0.1667 

0.9363 

0.3090 

16 

0.1667 

0.9363 

0.3090 

17 

0.0000 

0.3568 

0.9342 

18 

0.0000 

0.6667 

0.7454 

19 

-0.3090 

0.1667 

0.9363 

21 

-0.5774 

0.5774 

0.5774 

22 

-0.2697 

0.7697 

0.5787 

23 

-0.5787 

0.2697 

0.7697 

24 

-0.7697 

0.5787 

0.2697 

25 

0.5774 

0.5774 

0.5774 

26 

0.2697 

0.7697 

0.5787 

27 

0.5787 

0.2697 

0.7697 

28 

0.7697 

0.5787 

0.2697 

30 

-0.7454 

0.0000 

0.6667 

34 

0.7454 

0.0000 

0.6667 

37 

-0.3568 

0.9341 

0.0000 

41 

0.3568 

0.9341 

0.0000 

The  planner  selected  viewpoint  5  as  the  least  foreshortening  viewpoint. 

We  implemented  two  near  best  case  inspection  plans  and  two  near  worst  case  inspection 
plans.  The  planner  predictions  and  measurements  are  for  a  point  near  the  center  of  each 
Face. 


In  general,  the  predictions  for  the  best  case  inspections  plans  are  within  15%  of  the  predic¬ 
tions.  The  predictions  for  the  worst  case  inspections  plans  are  not  as  accurate  with  errors 
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running  up  to  30%.  The  reason  for  the  difference  in  accuracy  is  the  very  nature  of  the  best 
case  versus  worst  case  plans.  The  best  case  inspection  plans  produce  a  small  amount  of  error 
when  there  is  a  disturbance  in  intensity.  We  have  been  considering  this  disturbance  to  be 
from  the  light  intensity  variance  of  the  CCD  and  digitizer.  The  best  case  plan  has  a  low  sen¬ 
sitivity  to  this  noise,  while  the  best  case  plan  has  a  high  sensitivity.  Our  chalk  surface  is  not 
perfectly  lambertian.  So,  there  discrepancies  between  the  intensities  that  our  planner  pre¬ 
dicts  and  our  measurements.  Since  our  worst  case  plan  has  a  higher  sensitivity  to  these 
intensity  deviations,  the  accuracy  of  these  plans  is  worse.  This  observation  makes  us  con¬ 
clude  that  less  sensitive  plans  will  also  produce  more  accurate  results  when  there  are  devia¬ 
tions  from  the  expected  photometric  model. 


Planner 

Predictions 

Measurements 

Face 

Light 

(^(eerr)  ' 

(^(0err)  " 

(center  of  face) 

Sources 

Degrees 

Degrees 

A 

1,12,6 

4.05 

4.71 

B 

1,12,6 

4.11 

4.97 

A 

1,12,5 

1.23 

1.42 

B 

1,12,5 

1.22 

1.35 

A 

2,11,6 

1.14 

1.25 

B 

2,11,6 

1.15 

1.26 

A 

2,11,5 

4.22 

3.05 

B 

2,11,5 

4.23 

3.51 

The  deviation  that  we  measured  between  the  chalk  and  the  ideal  lambertian  model  is  shown 
below. 


Fig.  42.  Chalk:  Intensity  Versus  Viewing  Angle.  (Incident  Angle  =  0°  ) 

A  lambertian  surface’s  intensity  should  be  viewer  independent.  The  chalk  exhibits  some 
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reduction  in  intensity  with  increased  viewing  angle.  This  caused  the  brightness  predicted  by 
the  planner  to  be  greater  than  measurements.  The  next  table  lists  predicted  brightness  versus 
measured  brightness  near  the  center  of  each  face. 


Planner 

Prediction 

Measurement 

Percent 

Error 

Face 

(center  of  face) 

Sources 
in  Group 

Light 

Source 

I 

I 

A 

1,  12,6 

1 

61.0 

57.4 

-5.9 

A 

1,12,6 

12 

165.3 

150.0 

-9.3 

A 

1,12,6 

6 

119.0 

113.1 

-5.0 

B 

1,  12,6 

1 

165.3 

147.3 

-10.9 

B 

1,12,6 

12 

61.0 

52.7 

-13.6 

B 

1,12,6 

6 

119.0 

109.8 

-7.7 

A 

1,12,5 

1 

61.0 

57.4 

-5.9 

A 

1,12,5 

12 

165.3 

150.0 

-9.3 

A 

1,12,5 

5 

119.0 

99.2 

-16.6 

B 

1,12,5 

1 

165.3 

147.3 

-10.9 

B 

1,12,5 

12 

61.0 

52.7 

-13.6 

B 

1,  12,5 

5 

119.0 

97.0 

A 

2,  11,6 

2 

66.8 

64.4 

-3.6 

A 

2,  11,6 

181.1 

-5.6 

A 

2,  11,6 

6 

124.0 

-4.8 

B 

2,  11,6 

2 

181.1 

164.4 

-9.4 

B 

2,  11,6 

11 

66.8 

55.3 

-17.2 

B 

2,  11,6 

6 

130.3 

120.2 

-7.8 

A 

2,  11,5 

2 

64.4 

62.1 

-3.6 

A 

2,  11,5 

11 

174.6 

164.8 

-5.6 

A 

2,  11,5 

5 

125.7 

104.8 

-16.6 

B 

2,  11,5 

2 

174.6 

158.5 

-9.2 

B 

2,  11,5 

11 

64.4 

53.3 

-17.2 

B 

2,  11,5 

5 

125.7 

102.4 

-18.5 

We  measured  a  10X10  array  of  points  on  each  face  of  the  concavity.  This  allows  us  to  com¬ 
pare  the  predicted  and  measured  error  surfaces,  3a(0en.),  Fig.  39  and  Fig.  41. 

We  are  also  able  to  compare  the  predicted  and  measured  intensity  profiles  for  each  face.  The 
shape  of  the  two  surfaces  is  very  close.  As  expected,  the  measured  intensity  surface  is  atten¬ 
uated,  Fig.  40  and  Fig.  42. 


Fig.  43.  Best  Plan  Error  Surface,  Face  B,  90°  Chalk  Concavity,  Sources;  1,  12,  5 


Planner  Measurement 

Fig.  44.  Best  Plan  Intensity,  Face  B,  90°  Chalk  Concavity,  Sources:  1,  12,  5 


Planner  Measurement 

Fig.  46.  Worst  Plan  Intensity,  Face  B,  90°  Chalk  Concavity,  Sources:  2, 11,  5 
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An  intensity  image  of  the  concavity  and  a  needle  map  of  the  concavity  is  shown  in  Fig.  43. . 


Fig.  47.  Intensity  Image  from  Source  1,  and  needle  map  from  sources:  1,  12,  5 


page  54 


11.  Conclusions 

We  have  developed  an  illumination  planner  for  convex  lambertian  objects  and  for  simple 
lambertian  concavities.  The  planner  determines  how  to  position  light  sources  around  an 
object  so  that  we  illuminate  a  specified  set  of  faces  in  an  efficient  manner.  Using  a  noise 
model  of  the  CCD,  a  CAD  model  of  the  object  we  are  inspecting,  and  the  lambertian  reflec¬ 
tance  model,  the  planner  performs  a  statistical  simulation  to  determine  how  much  uncer¬ 
tainty  we  can  expect  in  our  shape  measurement  for  a  given  light  source  configuration.  The 
planner  generates  an  illumination  plan  that  illunfinates  the  specified  set  of  object  faces  while 
having  a  minimum  amount  of  uncertainty.  We  have  verified  that  the  illumination  plans  gen¬ 
erated  by  the  planner  are  valid  and  accurate. 

This  work  has  potential  applications  to  inspection  problems  in  industrial  environments. 
Although  the  photometric  stereo  method  requires  careful  calibration.  We  have  shown  that  it 
is  possible  to  accurately  predict  measured  shape  uncertainty.  This  ability  can  be  used  to  reli¬ 
ably  detect  defects. 
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